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[57] ABCTRACT 

An interactive video system employs Motion Picture 
Expert Group (MPEG) video compression to transfer 
images from a remote server to a television. The images 
correspond to dialog frames in a graphical user inter- 
face. During an authoring process, the dialog frames are 
created by first creating a background image and then 
adding foreground elements, such as buttons. A set of 
MPEG video frames is created by encoding the result- 
ing images according to the MPEG algorithm. The 
MPEG video frames are delivered to the television in 
sequence, where an MPEG decoder uses them to recon- 
struct the dialog images that are subsequently displayed. 
The system also contains an object-oriented database 
that maintains the necessary MPEG file ordering and 
also carries out menu navigation commands received 
from the user. The object classes include NODE, 
BRANCH, and DISPLAY. NODE objects correspond 
to dialog frames, and BRANCH objects correspond to 
user-selectable features such as buttons. DISPLAY 
objects correspond to either complete or partial images, 
as described in corresponding MPEG files. Data and 
function elements within the objects allow the system to 
track the user's navigation commands, and to highlight 
and unhighlight selectable features as navigation pro- 
ceeds in order to provide visual feedback to the user. 

13 Claims, 6 Drawing Sheets 
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receiving and interpreting subscriber input to ascertain 
RE MOTE DISPLAY OF AN IMAGE BY the subscriber's request. 

TRANSMii i iNG COMPRESSED VIDEO FRAMES Graphical user interfaces are particularly common on 
REPRESENTING BACKGROUND AND OVERLAY • personal computers (PCs) and workstations, which 
PORTIONS THEREOF 5 generally have in them all of the graphic software and- 

/or hardware required to perform the drawing tasks 
FIELD OF THE INVENTION required by the interface. In contrast, graphical user 

The invention relates generally to the fields of video interfaces are not as widely used for home information 
graphics and video transmission, and more particularly services. Such interfaces require both a considerable 
to interactive video systems such as modem video on amount of storage, for graphical libraries, drawing soft- 
demand systems. ware, etc., and also high information bandwidth be- 
. ^r^^^^TT^r^ tween the graphics hardware and the display. These 
BACKGROUND OF THE INVENTION f^^^ are r««dUy avaUable in a PC without unduly 
Information service to the home is a new field, en- increasing its overall cost. However, the addition of 
abled by the availability of storage and transmission graphics software and/or hardware to a relatively inex- 
technologies that can store and deliver data such as pensive settop greatly increases its cost. For this reason, 
video and images at an affordable cost. One example of graphics hardware and software either are not provided 
such a new service is video on demand, wherein a sub- in settops, or, when they are, have only limited func- 
scriber interacts with a remote video service to request tionality in order to minimize cost, 
the presentation of a particular movie or other video 20 in the cable TV industry, dedicated channels pres- 
program on the subscriber's television. This interaction ently deliver program guide background to the settop, 
may be via a telephone or may be via the same settop relevant text is then overiaid on the background to 
terminal and cable that are used to bring cable television display program guides with some graphics, such as 
programming to the television. ^nd colors. It is known to use graphical enhance- 
One technology that contributes to miproved home 25 ^^^^ integrated circuits in the settop to provide shad- 
information service is video compression technology. ^^^^^ enhancements to the displayed 
Usmg video compression, a motion-picture video c^ be ^ However, this method is limited in that there is 
sent over a commumcations chamiel usmg only a frac background, and the analog nature of the 
tion of the bandwidth of a conventional television chan- ^ background Umits the amount 
neL As a result, many more video channels can be car- 30 ^^^^^ ^^^ ^^ ^^^^ ^ ^^^^ Furthermore, this 
ned over a given medimn, such as modem television ^^^^^ ^^^^ overlays, and therefore can- 
cable media, so that it is now possib e to have pnvate or produce a robust graphical user interface of the type 
semi-pnvate video channels entenng the home, m a ""tpiL^uu^caiwuii^^Lgi^pm^ai u»ci miciia^^ji tiic t>pc 

manner similar to a home's telephone service. ^® ^™ ° above. 

One known technique for video compression is re- 35 SUMMARY OF THE INVENTION 

ferred to as the Motion Picture Industry Group t i- »• r ^ ^ - ^ ^ 

(MPEG) compression algorithm. In this algorithm, ^^S^* the hmitaUons of present-day mteracbve 

each frame of a motion-picture video is described either ^^f? ^yj^«=]^ ^ f ^^^^ °f the present mvention to 

independently or as a change from a previously-dis- ^^^^^^ ^^P^^^ ^ ^^^^ graphical user mterface m 

played frame. Thus a video scene might be described by 40 ^ interactive video sys'tem. It is a further object to 

a single independent frame which shows the entire «^able the display of high quahty graphics m connec- 

scene as it initially appears, followed by a long series of ^® graphical user interface, so that detailed 

change frames which describe the changes in the scene user-friendly menus can be displayed for easy use 

as actors move, for example. Usmg such a technique, by a TV subscriber. It is also an object to provide such 

considerable channel bandwidth can be saved by elimi- 45 capability at a low cost, so that the resulting interactive 

nating the redundant transmission of unchanging ele- system can enjoy wide use in the commercial video 

ments of the scene. entertainment industry. 

The MPEG algorithm and similar algorithms are In keeping vAth the above-mentioned objectives, the 
ah-eady in use in video information services. The present invention is in one aspect a method for display- 
MPEG algorithm is described in the following specifi- 50 ing images from a remote location without the necessity 
cations: ISO/EEC 11172, Nov. 1991 (MPEG 1); and of storing images or drawing software on the settop. 
ISO/IEC WGll N0501, July 1993 (MPEG 2). Both of Rather, the images are stored at the remote video ser- 
these specifications arc incorporated herein by refer- vice location and transmitted therefrom in a com- 
encc. Also, C-Cube Inc. sells. MPEG decoder chips pressed format. A video decoder already used in the 
having part nos. CL450 and CL950. 55 settop for receiving motion picture videos, such as the 

While video compression is thus enabling the creation above-mentioned MPEG decoder, is also used to recon- 

of a new video transmission network, other techniques struct the still images, such as menus, employed in the 

are being used to enable greater subscriber interaction graphical user interface. 

with the video service. One method for enabling such In particular, this aspect of the invention includes the 

interaction via the cable television system is to employ 60 method steps of 

a series of graphical menus that are displayed on the TV (i) creating a background image to serve as the back- 
screen. The subscriber interacts with these menus in a ground in the still image; 

fashion similar to a computer user's interaction with (ii) adding foreground elements to the background 

so-called graphical user interfaces, which employ win- image to compose the still image; 

dows, dialog boxes, buttons, pull-down menus, and 65 (iii) encoding the background and still images according 

other features to present information and options to the to a compression algorithm that is capable of describ- 

user. The remote video service is responsible for con- ing a single image by an independent video frame and 

trolling the display of these items on the screen, and for also capable of describing the differences between a 
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pair of images by a change video frame, the result of played dialog image that are necessary to unhigh- 
the encoding step being a compressed independent light the control object that is currently high- 
video frame independently describing the back- lighted; 

ground image and a sequence of at least one com- (ji) selecting another of the control objects for high- 
pressed change video frames describing the differ- 5 lighting as directed by the one navigation com- 
ences between the background unage and the still mand; and 

(iii) transmitting over the communications medium a 

(iv) storing the compressed video frames on a computer change frame representing changes to the dialog 
coupled to the conmiunications medium in a manner ^^at are necessary to highlight the selected 
sufficient to preserve the sequentiality of the com- 10 control object* 

pressed change video frames; ^ ^ . and (vi) invoking the appUcation program associated 

(v) rctnevmg the stored compressed video frames m cuixently-highlighted control object if the 
preparation for their subsequent transmission on the keystroke identifier represents a select command to be 
commumcations medium; and computer. 

(vi) sequentially transmittmg the retneved compressed 15 f c^x. • *• * i j * * 

.'y^,^-'., - * ^. r _ This aspect of the mvention takes advantage of video 

mdependent video frame and the retneved com- : ^ ^ ^ • . t.- 

press^ change video frames on the communications <^f^P~ jo cany out robust navigaUon m a ^aphi- 

medium, each retrieved frame being transmitted in a "^^^'f^^^ "^^^ consetvmg video bandwidth on 

corresponding packet having a header that enables commmucation chamiel, thus eoabhng its use with 

the subsequent identification of that packet by a video 20 ^umerousr subscnbers over a shared physical channel, 

display device coupled to the communications me- ^9 ^ "^vendon is a meUiod of 

dium upon which the still image is to be displayed. carrymg out navigation m a graphical user mterface by 

By employing compressed video frames, in particular ^® °^ * particular type of object-oriented database. The 

compressed independent frames and change frames, database objects are (1) a node object corresponding to 
high quaUty graphics can be achieved using a reason- 25 ^ dialog video image; (2) branch objects each corre- 
able amount of signal bandwidth. Therefore many sub- sponding to a particular control object appearing on the 
scriber interactions can take place simultaneously with- node object; and (3) display objects each corresponding 
out sacrificing an undue number of video chaimels in to one of the many discrete images that must be em- 
the broadcast video system. Furthermore, the cost of ployed in the graphical interface. For example, a branch 
the additional settop hardware required to process these 30 object can correspond to a "button" whose appearance 
frames is significantly less than the cost of a fiill-blown changes when it is selected. One display object can be 
graphics processor. Therefore the graphics capabihty of used to display the image having die button not se- 
the settop has been significantly improved without a lected, and another for the image having the button 
commensurate increase in its cost, especially since the selected. The node, branch, and display objects have a 
same video decoding hardware server "double duty" to 35 variety of data and fimction elements to enable them to 
achieve a major part of the present invention's fimction- carry out initialization, display, and navigational steps, 
ality. and also to invoke application programs associated with 

In another aspect, the invention is a method of carry- some dialog control objects, 
ing out navigation in a graphical user interface by em- 
ploying independent and change video frames. This 40 BRIEF DESCRIFnON OF THE DRAWINGS 
aspect includes the steps of An embodiment of the invention will be described 

(i) instantiating apphcation programs each being associ- reference to the accompanying drawings, in 

ated with a corresponding one of control objects which: 

appearing on a dialog video image to be subsequently pjQ t is z block diagram of an interactive video 
displayed on a display device coupled to a communi- 45 system embodying the principles of the present invcn- 
cations medium comiected to the computer; ^^q. 
Cii) transmitting over the communications medium a pjQ 2 is a flow diagram showing how video frames 
sequence of at least one cotnpr^ video frame identifymg information are created and stored 

coll^tively represcntmg an unhighhghted vem^^ of ^^^^ components of the system of FIG. 1; 

die dialog video miage. the transmitted video frames 50 3 ^ ^ ^^^^ 

being encoded accordmg to a compression algorithm ^ 
that is capable of descnbmg a smgle image by an tt h th ' 

independent video frame and also capable of describ- settop s&owntaerem; , . 

ing the differences between a pair of images by a , * ^ Aow diagram showmg a method of creat- 

change video frame* 55 "^8 a set of unage files for a menu type of video frame 

(iii) transmitting over the communications medium a ^^^^^ ^^^^^^^ ^ highlighted and unhigh- 
change video frame representing changes to be made ^ShVt^ ^ ^ navigates around the menu; 

to the unhighUghted dialog video image in order to ^ shows a background image created by the 

highlight an initial one of the control objects appear- method of FIG. 4; 

ing thereon; 60 ^O. 6 shows a complete, unhighlighted dialog image 

(iv) awaiting the receipt of a keystroke identifier from consisting of the image of FIG. 5 with additional but- 
tiie communications medium; tons overlaid thereon; 

(v) executing the following navigation steps if the re- 7 shows the dialog image of FIG. 6 with one 
ceived keystroke identifier represents one of a set of button highlighted to indicate that it is the currentiy 
navigation commands to be carried out by the com- 65 active button; and 

puter: FIG. 8 is a diagram of an object-oriented implementa- 

(i) transmitting over the communications medium a tion of a dialog database in a server within the system of 
change frame representing changes to the dis- FIG. 1. 
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DETAILED DESCRIPTION 



^^The~6verlay images 32 occlude the portions of the 
background image 3O m^eT;^^!g0^0^^i^^di and may 



In FIG. 1, an authoring station 10, video/image in fact occlude other overlay images in whole or in part', 
server 12 and gateway server 14 are coupled together An example of such a case is a conventional "pop-up'^ 
via a communications link 15 such as Ethernet The 5 menuy which' wK^h it apj^^^^ 

servers are at a **remote" location, such as a cable televi- played overlay image. The overall dialog image 28, 
sion transmission facility. The servers 12 and 14 are then, can be thought of as being created from a parties- 
connected via one or more conamunications media 16 to lar se^uettbe 6f overlay images 32 placed on top of tie 
settops 18 in subscribers' homes, each settop 18 being in ^background image 30. This layered nature of the dialog 
turn connected to a corresponding TV 20. There is also 10 image> in particular^ is advantageously exploited in the 
a remote controller 22 associated with each TV 20. presem mventiok However, the princip^ 
Appearing on the communications media 16 are two tion are genefadly Applicable to any still image which 
independent data streams: a unidirectional video data can^^^»^d6oomposed into a background image and a 

stream 24 being broadcast from the video server 12 to seqiiciiGl^f overlay images, _ _ . J 

the settops 18, and a bidirectional control data stream 26 15 ^ ^As^will be described in greater detail below with 
carrying control messages between the gateway server reference to FIG. 3, the components shown in FIG. 1 
14 and the settops 18. It may be that the communica- cooperate to transmit the dialog image 28 to the TV 20 
tions medium 16 is a single medium such as a conven- for display. In particular, data representing the individ- 
tional television cable; in such a case, the two data ual images 30, 32 are transmitted to the settops 18, 
streams 24 and 26 are multiplexed thereon in any of 20 where a reconstruction process is carried out that re- 
several known fashions. In alternative embodiments, the suits in the display of a reconstructed dialog image 28' 
data streams 24 and 26 may be carried on separate phys- made up of a reconstructed background image 30' and 
ical media, or may intermittently exist on the same me- reconstructed overlay images 32'. Any text appearing 
dia, such as in the case of a packet-switched network. on the image is also reconstructed and displayed. 
While the particular mode of transmission is not ger- 25 The settop 18 performs the conventional function of 
mane to the present invention except as hereinbelow receiving video frames from the video data stream 24 
described, it should be noted that the U.S. Federal and converting them into an analog TV signal, such as 
Communications Commission has recently allocated a an NTSC signal, suitable for direct use by the TV 20. 
band of frequencies in the range of 500-800 MHz for The settop 18 must of course contain de-compression 
digital television transmission. It is anticipated that the 30 circuitry that is compatible with the compression tech- 
present invention will find application in cable broad- nique being used in the system. In the case of an MPEG 
cast systems employing this newly-allocated frequency type of video data stream, the settop 18 includes an 
band, MPEG decoder chip of the type mentioned above, and 

The partition of the server function into video/image additional conventional circuitry to receive MPEG 
12 and gateway 14, as shown in FIG. 1, has the advan- 35 video frames and provide them to the decoder, as well 
tagc that each server workstation can be independently as a conventional frame buffer and associated circuitTy 
optimized for maximum performance. For example, the for translating its contents to the TV-compatible analog 
video/image server 12 has high storage capacity, and signal. The settop 18 also receives a signal from the 
must deUver data to a unidirectional channel at very remote 22 whenever a key is pressed, and transmits a 
high bandwidths. The gateway server 14, in contrast, 40 message on the control data stream 26 that identifies the 
carries out bidirectional conamunications and other keystroke. 

functions. While it is possible, and indeed less expensive. Referring now to FIG. 2, the procedure followed at 
to carry out both functions in a single workstation, the the authoring station 10 is described. For each frame to 
available video bandwidth in such an embodiment be created, where "frame" refers to a complete still 
would be less than that of the illustrated configuration, 45 image to be displayed, the background image 30 of FIG, 
due to interference between the video and communica- 1 is created at step 40. This image, as with all the irqages 
tions activities. Therefore a partition of the type shown created in FIG. 2, can be created using a conventional 
is preferred in the near term. drawing program that outputs a bitmap representation 

At the authoring station 10, which may for example of the image. Examples of such drawing programs are 
be a conventional graphics workstation, a dialog image 50 CorelDraw from Corel, Inc.; Windows SDK from Mi- 
28 is created by a process described in greater detail crosoft. Inc.; and MediaClips from Aris Entertainment 
below with reference to FIG. 2. The dialog image 28 is Co. Microsoft Viewer may also be used to convert 
part of a graphical interface employed in the video image files from one format to another if necessary. The 
system to enable a subscriber to interact therewith. For background image 30 may also be captured from a pho- 
example, it may be a conventional menu showing lists of 55 tograph, paused video, or other graphical source using 
available command options, such as commands for re- conventional image capturing hardware and/or soft- 
questing the presentation of a movie, etc. The dialog ware, such as a scanner, 

image 28 consists of a background image 30 and one or Once the background image 30 is created, a unique 
more overlay images 32. An example of an overlay ID or name is given to it at step 42, The bitmap repre- 
image is a conventional dialog box which displays a 60 sentationof the background image 30 is then encoded in 
short message to the subscriber on a small portion of the step 44. The encoding is carried out according to the 
screen; another example is a conventional control **but- aforementioned MPEG algorithm by providing the 
ton" that is used to initiate a particular action associated image Hies to MPEG encoding software also installed 
therewith. The dialog image 28 also typically has blocks on the system. This encoding software may be obtained 
of text, not shown in FIG. 1, appearing on the back- 65 from a company called Portable Video Research Group 
ground image 30 and/or the overlay images 32. A more in California. The MPEG algorithm is capable of de- 
rcalistic image is described below with reference to scribing an image by either a single independent video 
FIGS. 5 through 7. frame, called an I-frame, or by a combination of an 
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initial I-&ame and one or more succeeding change file or in-memory array 86, which contains the &ame 

frames, called P-frames, each of which describes a IDs and attributes for the text objects 84. 

change to a previously-cUsplayed image. Typically, the The various final images are then encoded in step 44, 

P-frame data is written into a frame buffer whose con- and stored at the video server 12 at step 46. Similarly, 

tents are already being displayed, resulting in a modifi- ^ the parameter file 86 and text 84 are stored at the gate* 

cation to the displayed image. way server at steps 46 and 58. These stored data are 

The result of encoding the background image in step then transmitted over separate data streams to the set- 

44 is the creation of a compressed I-frame that indepen- top 18. The P-frames and I-frames stored on the video 

dently describes the background image 30. This back- server 12 are sent via the video data stream 24, while the 

ground I-frame is then saved, at step 46, in an image 10 text and control information stored on the gateway 

database 48 in the video/image server 12 of FIG. 1, and server 14 are sent via the control data stream 26, which 

the ID corresponding to the background I-frame is ^ referred to in FIG. 3 as a "gateway interface" 90. 

saved in a dialog database SO in the gateway server 14. At the settop 18, a frame display controller 92 reads 

After the above-described steps have been carried out received parameter file 94 to obtain the frame IDs of 

for the background image 30, various overlay images images being sent in the video data stream 22. At 

are created in step 52. These may be dialog boxes, but- ^® controller grabs each frame in the 

tons, and/or icons as mentioned above. The result of ^d«> f?"^ 24 that matchw one of tiie IDs in the 

this step is a nearly complete dialog image 28, missing parameter file 86, and feeds it to the MPEG decoder 98. 

only any necessary text. The overlay images are as- ^} ^^^P ^ ^ reconstructed images from the decoder 

signed IDs in step 54, and then fed to the MPEG en- ^he rej^ived text 102 to create the 

coder in step 44. The result of this encoding is a set of ^ reconstructed miage 28 . which is then displayed at 

compressed P-frames, each describing a corresponding ^, , .t_j. .-, . • 

one of the overlay images 32. JJTiat has so far been descnbed is a displ^^^^^ 

In step 46. the compressed P-frames and their associ- „ °° '"'I f imique IDs associated with the 

^ , 1 J • *!. -J /•« ^-^ 25 Video frames to enable the settop 18 to grab them as 

ated IDs are also saved m the video/miage server 12 . t* u tJ v * j tf *u * i* 

, ^ -D r J T £_ they amve. It should be noted, however, that altema- 

and gateway server 14. The P-frames and I-frame are embodiments may not require suci IDs. In an 

saved as an ordered list, so that they may be transmitted ^^^^ ^ '^^^ ^^^^ j^^^ 

m the proper sequence to ensure correct recon^^ ^ embedded *^dware ID" that identifies the 

of the miage. This bst can take the form, for example, of 3^ 3 ^^^^ j^^^ ^^^^ 3 

a convenuonal hnked hst data structure. ^ ^^^^ ^ transmission format, separate IDs can be 

At step 56, text is assigned to locations on the frame. dispensed with, because the settop 18 simply grabs 

The text is given several attnbutes, such as size, loca- ^^^^^ ^^^^ containing its hardware ID. 

Uon, font, etc. This information is also saved, at step 58, ^hilc the ovcr^ system environment and general " 

m the dialog database 50. In the absence of any parent/- 35 operation of an embodiment of the present invention 

child frame relationships, descnbed below, the descnp- i^^yg y^^^ described with reference to FIGS. 1 through 

tion of the dialog frame 28 is then complete and ready 3^ remaming figures show an exemplary use of the 

for subsequent transmission and display. system, as well as a particular implementation of the 

If invoking a control or button on one frame is in- server dialog database 50 of FIG. 2. FIG. 4 shows a 

tended to cause another dialog frame to appear, then ^ methodof creating a particular set of images for a menu 

both of these frames must be created and logically video frame, and FIGS. 5 through 7 show some of the 

linked together. In such a case, the first frame is referred resulting images. The files representing these images are 

to as the "parent", and the second as the "child". After saved in the server image database 48 of FIG. 2. FIG. 8 

saving a parent frame at step 58, the process takes the no shows the above-mentioned implementation of the 

branch at step 60, the "yes" branch at step 64, and re- 45 server dialog database 50. 

peats the foregoing steps to create the child frame. Referring now to FIGS, 4 through 7, a background 
Upon completion of the child frame, the process takes image is created in step 1 10. The resulting image, resem- 
the "yes" branch at step 60, a linkage is saved at step 62 bling a blimp flying at the top of the screen, is shown in 
to establish the parent/child relationship, and then the FIG. 5. This image is saved as a bitmap or other graph- 
process is complete. An example of how such a parent/- 50 ics file named IMAGEl. At step 112, a button is added 
child linkage is created is given below with reference to ©r overlaid on the background image; the resulting 
FIG. 8. At the completion of the process of FIG. 2, image is shown in FIG. 6. This image is saved and 
then, all of the necessary MPEG frames, text, and con- called IMAGE2. At this point IMAGE2 represents the 
trol information necessary to describe the dialog image complete, unhighlighted menu frame. While in this 
28 has been created and stored on the servers 12 and 14 55 simple example only one overlay was used in creating 
of FIG. 1 for subsequent transmission to the settops 18 the final image, more complicated images may of course 
for reconstruction. employ more overlays as needed; in such a case, addi- 
FIG. 3 shows the overall operation of the system of tional image files each representing successively-more- 
FIG. 1 in a dataflow format Initially, objects such as complete images are created. 

background objects 70, control objects 72, image over- 60 The button labelled "OPTIONS ..." in FIG. 6 is 

lay objects 74, and text objects 76 are picked and assem- intended to cause another menu screen to appear when 

bled into the dialog image 28 by the image preation pressed. This is an example of parent/child frames as 

steps 40, 52, and 56 of FIG. 2. These objects may be discussed above. 

stored in a graphics library on the authoring station 10, In step 1 14, a set of additional image files is created, 

for example. Tliis authoring process results in final ver- 65 each describing an image showing a corresponding 

sions of the background image 78, the controls 80, the button highlighted. This is achieved by importing IMA- 

other image overlays 82, and text 84. Additionally, GE2 into the drawing tool, adding the highlighting 

control information is created and placed in a parameter effect to the desired button, and then saving the result- 
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ing image. An example is shown in FIG. 7, in which the each full menu display. Also, two or more NODE ob- 

button labelled "START" is highlighted. This step jects may be linked together in the above-described 

results in the creation of image files IMAGES through • parent/child relationship to create more complicated 

IMAGEn, where n— 2 is the number of buttons. menu patterns. 

While in FIG. 7 highlighting is shown as enhance- 5 The data and functions of NODE objects are shown 

ment of the outline of a button, it may be achieved in in NODE object 120 and are as follows: 

other ways. For example, the object to be highlighted PARENT is a pointer to the NODE object through 

could instead have its colors changed, or be displayed in which the current NODE object was accessed. For 

inverse- video. These and other suitable highlighting example, NODE object 122 is a child of NODE object 

techniques are known in the art 10 120, and so its PARENT element contains the ID of the 

At this point, the complete set of images necessary to parent NODE object 120. 

display and navigate the menu is complete. At steps 1 16 INITIAL OVERLAY SEQUENCE is a pointer to 

and 118, these image files are encoded according to the the ordered set of MPEG I and P frames that comprise 

above-mentioned N/PEG compression algorithm for the initial unhighlighted menu display. An example is 

later transmission. In step 116, an MPEG I-frame 15 the above-mentioned set of files I—IMAGEl and 

LJMAGEl is created from IMAGEl, and an MPEG P_IMAGE2. INITLU. OVERLAY SEQUENCE 

P-frame P_IMAGE2 is created from the sequence points to the first of a hnked list of DISPLAY objects 

(IMAGEl, IMAGE2) . In step 1 18, additional P-frames referring to these frames, such as the DISPLAY object 

are created to aUow highlighting and unhighlighting of pointing to the file I_IMAGE1. A list is readily created 

each button. For each pair of files (IMAGE2, IMA- 20 by using NEXT DISPLAY OBJECT to point to the 

GEO, 2 P-frames are created: PH-JMAGEi describes next DISPLAY OBJECT in the list. Alternatively, 

going from unhighlighted to highlighted, and INITIAL OVERLAY SEQUENCE may point to a 

PU-_IMAGEi describes going from highlighted to un- separate LIST object, not shown in FIG. 8, that con- 

highlighted. This step results in the creation of image tains an ordered list of pointers to those DISPLAY 

files PH_IMAGE3— PH_IMAGEn and 25 objects. While use of a general-purpose LIST object 

PU_IMAGE3 — PU_IMAGEn. As stated above, all of entails more overhead, it also enables the creation of 

these files are saved in the server image database 48 for other Hst typeS that may be useful for other operational 

subsequent use. aspects of the database. . 

FIG. 8 describes an object-oriented implementation INITIAL TEXT points to an initial one of an or- 

of the server dialog database 50 of FIG. 2, and its opera- 30 dered list of TEXT objects appearing on the unhigh- 

tion upon images such as that of FIG. 6. Four object lighted menu display. 

classes are defined, namely NODE, BRANCH, DIS- FULL FRAME points to a single DISPLAY object 

PLAY, AND TEXT. In FIG. 8, objects 120 and 122 are that refers to the MPEG I-frame that describes the full 

oftheNODEclassjobjectsl24, 126, andl28areofthe unhighlighted display. In the example of FIGS. 5 

BRANCH class; object 130 is of the DISPLAY class; 35 through 7, this is IMAGE2. 

and object 132 is of the TEXT class. Each object of a FOCUS keeps track of which one of several 
given class encapsulates the data and functions shown in BRANCH objects is active at a given time, and INI- 
an exemplary instance in FIG. 8. For example, NODE TIAL FOCUS is the first BRANCH object to be high- 
objects 120 and 122 each have a function INITIAL- lighted when the NODE object is first activated. 
IZATION as well as data elements PARENT, INI- 40 The above-described NODE data elements are used 
TIAL OVERLAY SEQUENCE, INITL^L TEXT, . by the NODE'S INITIALIZE function to bring the 
FULL FRAME. FOCUS, AND INITIAL FOCUS. display to an initial state. The INITIALIZE function 
The use of these various functions and data elements is sends the MPEG files of INITIAL OVERLAY SE- 
described below. QUENCE to the settop 20, and then calls the HI- 
The function ofthc database ofFIG. 8 may be gener- 45 LIGHT function of the BRANCH object that is 
ally understood as follows. Each NODE object 120, 122 pointed to by INITIAL FOCUS. INITIALIZE then 
describes a dialog screen or frame principally by point- sends the text elements from the TEXT objects pointed 
ing to associated DISPLAY objects. The screen con- to by INITIAL TEXT, and finally copies ESTITIAL 
tains buttons or other features that enable navigation FOCUS to FOCUS to establish ^e current branch 
around the dialog menu; FIG. 6 shows an example of 50 focus. 

such a screen. Each of these navigation features, or The function FIND-BRANCH is used as part of one 

buttons, is described by a corresponding BRANCH means of navigation. When a keystroke identifier is 

object, such as BRANCH object 122. The BRANCH received that corresponds to a response code, FIND— . 

objects follow the navigation commands of the user and BRANCH is called with the received response code as 

provide visual feedback by highlighting and unhigh- 55 an argument, FIND--BRANCH searches through the 

hghting display features as navigation progresses. BRANCH objects to find the one having a response 

BRANCH objects also may invoke other NODE ob- code matching the received response code. When it 

jects, for example to implement a nested menu. This finds a match, it unhighlights the current focus, changes 

relationship exists between BRANCH object 124 and the focus to the BRANCH object having the matching 

NODE object 122, for example. DISPLAY objects 60 response code, and then highlights the new focus. The 

contain the names of graphics files to be displayed, search performed by FIND—BRANCH is a tree search, 

pointers to TEXT objects, and pointers to other DIS- and any of several conventional search algorithms may 

PLAY objeqts to implement the N/PEG overlay be employed. If no matching response code is found, the 

scheme. Finally, TEXT objects contain data describing focus is not changed. 

associated text. 65 The BRANCH objects identify each selectable but- 
NODE objects either contain or refer to all of the ton in a NODE object and their navigational interrela- 
information that is needed to create and navigate a full tionships, and also contain navigation functions to en- 
dialog or menu frame. One NODE object is created for able a user to navigate the dialog menu represented by 
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the NODE object. The data and functions of BRANCH 
objects are shown in BRANCH object 124 and are as 
follows: 

HILIGHT DISPLAY OBJECT points to a DIS- 
PLAY OBJECT having the button corresponding to 
this BRANCH OBJECT highlighted. Similarly, UN- 
HILIGHT DISPLAY OBJECT points to a DISPLAY 
OBJECT having the button corresponding to this 
BRANCH OBJECT unhighlighted. In the preferred 
embodiment, each of these DISPLAY OBJECTS is 
pointing to a corresponding P-frame that describes only 
the highlighting or onhighlighting of the button, for 
example the above-mentioned iiles PH . TMAGE3 and 
PU_IMAGE3 respectively. 

UP-JBRANCH, DOWN-BRANCH, LEFT— 
BRANCH, AND RIGHT-BRANCH each point to 
corresponding BRANCH objects that are adjacent to 
this BRANCH object in the corresponding direction. 
These pointers control how the BRANCH focus is 
changed when an "arrow" key is pressed on the remote 
controller 22. These data elements may be programmed 
with null values if there is no corresponding BRANCH 
object, i.e., if the BRANCH object containing them is 
near an edge of the dialog image. Alternatively, these 
pointers may also be programmed to *Vrap around" the 
screen, so that for example the LEFT-BRANCH 
pointer for a leftmost BRANCH object on an image 
would point to the BRANCH object at the far right of 
the image. 

Fmally, DESTINATION NODE points to another 
NODE object to be displayed if the "select*' key on the 
remote controller 22 is pressed. If there is no such 
NODE object, then DESTINATION NODE contains 
a null value. 

The above-described BRANCH data elements are 
used by BRANCH'S various functions to navigate the 
menu and display the changing branch focus to the user. 
The HILIGHT function sends to the settop 18 the 
MPEG files of a DISPLAY object havmg the button 
associated with the BRANCH object highlighted. If an 
associated TEXT object exists, it also sends that Simi- 
larly, the UNHILIGHT function sends to the senop 18 
the MPEG files of a DISPLAY object having the but- 
ton associated with the BRANCH object unhigh- 
lighted, and also removes any TEXT objects that were 
previously overlaid by HILIGHT. 

The APPLICATION-CALL function is invoked 
when a branch object is selected and there is no destina- 
tion node, i.e., when DESTINATION NODE contains 
a null value. APPLICATION—CALL in turn invokes 
an application program, such as video-on-demand, ex- 
it/quit, etc., that is associated with the selected control 
object Navigation terminates when APPLICA- 
TION-CALL is invoked. 

The UP ARROW, DOWN ARROW, LEFT AR- 
ROW, and RIGHT ARROW functions carry out navi- 
gation. Each one first calls the UNHILIGHT of the 
current BRANCH OBJECT, and then changes the 
NODE'S FOCUS to the corresponding BRANCH, e.g., 
the BRANCH that is navigationally above, bdow, to 
the left or to the right of the current BRANCH. If there 
is no BRANCH in the indicated direction, then the 
current FOCUS is maintained. Then HILIGHT of the 
new FOCUS is called. 

The SELECT function is called when the "selecf ' 
button on the remote controller 22 is pressed. SELECT 
invokes the NODE object pomted to by DESTINA- 
TION NODE by calling its INITIAUZE function. 
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The DISPLAY objects contain data elements that 
identify associated MPEG files to be displayed for a 
NODE or BRANCH, associated TEXT objects, and 
linkage pointers to other DISPLAY objects. The data 
elements of DISPLAY objects are shown in DISPLAY 
object 130 and are as follows: 

ID is the above-referred-to unique ID that is created 
for each MPEG file. 

BITMAP FILE NAME identifies the bitmap file that 
is the source for the image to be displayed, for example 
IMAGE2. MPEG FILE NAME identifies the MPEG 
file (either I-frame or P-frame) that is to be transmitted. 
Examples are the files I_IMAGE1, P_IMAGE2, 
Ptt-IMAGEi, etc. PREVIOUS DISPLAY OBJECT 
and NEXT DISPLAY OBJECT are pointers linking 
this DISPLAY object to its neighbors in an MPEG 
frame sequence used to build up the overall image. 
TEXT OBJECT points to a text object having any text 
required for the display. 

The TEXT objects contain data elements that de- 
scribe text to be overlaid on an image. The data ele- 
ments of TEXT objects are shown in TEXT object 132 
and are as follows: 

TEXT STRING is the text string to be displayed. X 
COORDINATE and Y COORDINATE describe the 
placement of the text on the screen. COLOR, SIZE and 
FONT respectively indicate the color, size and font 
with which the text should be displayed. NEXT TEXT 
OBJECT is a pointer to a subsequent TEXT OBJECT 
in an ordered list of TEXT OBJECTS. 

While the foregoing has described two particular 
means of navigating around the menu, namely (1) by 
employing linked BRANCH objects and ARROW 
functions corresponding to arrow keys, and (2) using 
response codes and branch searching, the database of 
FIG. 8 may also be used with suitable modification 4n 
other navigational schemes. For example, if the user's 
TV system employs a cursor, a CURSOR— MAP func- 
tion can be included to map the cursor location to the 
nearest BRANCH object, which then becomies FO- 
CUS. The CURSOR— MAP function would be located 
in the NODE object, and would be invoked each time 
there was a change in the cursor position. A given sys- 
tem could supi>ort either cursor navigation, or arrow 
navigation, or both, and the implementation of the dia- 
log database 48 may change depending on the support 
provided. For example, if the database need only sup- 
port ctu^or navigation, then the various ARROW func- 
tions and BRANCH data elements in the BRANCH 
' objects would be unnecessary. 

It win be appreciated by those skilled in the relevant 
art that while the database structure shown in FIG. 8 is 
particularly suitable for use with the P-frame/I-frame 
methods of the current invention, it may also be used 
with other transmission and display schemes. For exam- 
ple, it may be used in a system that re-transmits the 
entire screen each time the image changes, rather than 
sending P-frames. In such a system, the linkages be- 
tween DISPLAY objects would be unnecessary; how- 
I ever, the remaining features would still be of valuable 
use. Accordingly, the database of FIG. 8 embodies 
additional aspects of the present invention that have 
independent merit 

In a similar vein* there are other database implemen- 
: tations that will support the I-frame/P-frame display 
technique described herein. These may include object- 
oriented databases having different class structures, or 
relational databases, as well as more traditional struc- 
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tures. All such embodiments are intended to be de- 
scribed by the claims appended hereto. 
What is claimed is: 

1. A method of transmitting a still image over a com- 
munications medium, comprising the steps of: 

creating a background image to serve as the back- 
ground in said still image; 

adding foreground elements to said background 
image to compose said still image; 

encoding said background image and said still image 
according to a compression algorithm that is capa- 
ble of describing a single image by an independent 
video frame and also capable of describing the 
differences between a pair of images by a change 
video &ame, the result of said encoding step being 
a compressed independent video frame indepen- 
dently describing said background image and a 
sequence of at least one compressed change video 
frames describing the differences between said 
background image and said still image; 

storing the compressed video frames on a computer 
coupled to said communications medium in a man- 
ner sufficient to preserve the sequentiality of said 
compressed change video frames; 

retrieving the stored compressed video frames in 
preparation for their subsequent transmission on 
said communications medium; and 

sequentially transmitting the retrieved compressed 
independent video frame and the retrieved com- 
pressed change video frames on said communica- 
tions medium, each retrieved frame being transmit- 
ted in a corresponding packet having a header that 
enables the subsequent identification of that packet 
by a video display device coupled to said commu- 35 
nications medium upon which said still image is to 
be displayed. 

2. A method according to claim 1, wherein said com- 
pression algorithm is the Motion Picture Industry 
Group (MPEG) algorithm. 40 

3. A method as set forth in claim 1, further compris- 
ing the steps of: 

examining the head^ of packets received by said 
video display device to identify those containing 
' the retrieved compressed video frames transmitted 43 
thereto; 

decoding the compressed video frame contained in 
each identified packet in accordance with said 
compression algorithm to reconstruct said back- 
ground image and said foreground elements for 50 
subsequent display on said display device; and 

displaying the reconstructed background image and 
foreground elements on said display device. 

4. A method as set forth in claim 1, fiirther compris- 
ing the steps of: 55 

creating an independent frame identifier and change 
frame identifiers, said independent frame identifier 
being associated with said stored compressed inde- 
pendent frame, and each of said change frame iden- 
tifiers being associated with a corresponding one of 60 
said stored compressed change frames; 

storing the created frame identifiers on said com- 
puter; 

retrieving the stored frame identifiers; and 
placing each retrieved frame identifier into the 65 
header of a corresponding packet containing the 
compressed frame with which that frame identifier 
is associated. 



5. A method according to claim 4, further comprising 
the steps of: 

transmitting the created frame identifiers to said 
video display device to enable it to identify com- 
pressed video frames contained in packets to be 
subsequently received thereby from said communi- 
cations medium; 

comparing the headers of packets received by said 
video display device with the frame identifiers 
transmitted thereto in order to identify those pack- 
ets containing the retrieved compressed video 
frames transmitted thereto; 

decoding the compressed video frame contained in 
each identified packet in accordance with said 
compression algorithm to reconstruct said back- 
ground image and said foreground elements for 
subsequent display on said display device; and 

displaying the reconstructed background image and 
foreground elements on said display device. 

6. A method according to claim 1, wherein said still 
image is a dialog video frame, and wherein said step of 
creating a background image comprises the step of 
creating a dialog box, and wherein said step of adding 
foreground elements comprises the step of adding 
graphical objects inside said dialog box that are part of 
a graphical user interface. 

7. A method according to claim 6, wherein said step 
of adding graphical objects comprises the step of adding 
a control button. 

8. A method according to claim 6, further comprising 
the steps of: 

creating a text block to appear on said dialog video 
frame, said text block mcluding a text string and 
having descriptive text attributes associated there- 
with; 

storing said text string and said descriptive text attri- 
butes on said computer; and 

transmitting said text string and said descriptive text 
attributes over said communications medium to 
enable said video display device to reconstruct said 
text block for subsequent display thereon. 

9. A method according to claim 8, further comprising 
the steps of: 

. reconstructing said text block from the text string and . 
descriptive text attributes transmitted to said video 
display device; and 
displaying the reconstructed text block on said video 
display device. 

10. A method according to claim 1, wherein each of 
said transmitting steps comprises the step of transmit- 
ting the corresponding data on a broadcast communica- 
tions medium interconnecting said computer and said 
video display device. 

11. A method according to claim 10, wherein said 
broadcast communications medium is a cable television 
network. 

12. A method according to claim 1, wherein said step 
of transmitting independent and change frames com- 
prises the step of transmitting said frames on one com- 
munications medium interconnecting said computer and 
said video display device, and said step of transmitting 
frame identifiers comprises the step of transmitting said 
identifiers on another communications medium inter- 
connecting said computer and said video display de- 
vice. 

13. A method of displaying a dialog video frame on a 
display device from a remote location coupled to said 
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display device by a broadcast communications medium, 
comprising the steps of: 

creating a background image to serve as the back- 
ground in said dialog video frame; 

creating overlay images to be overlaid on said back- 
ground image to serve as foreground portions of 
said diaJog video frame; 

creating a text block to appear on said dialog video 
frame, said text block including a text string and 
having descriptive text attributes associated there- 
with; 

encoding said background and overlay images ac- 
cording to a Motion Picture Expert Group 
(MPEG) algorithm to create a compressed MPEG 
I-frame independently describing said background 
image and compressed MPEG P-frames each de- 
scribing a corresponding one of said overlay im- 

storing said I-&ame and said P-frames on a video 
server at said remote location; 

storing said text string and said descriptive text attri- 
butes on a gateway server at said remote location; 

creating an I-frame identifier and P-frame identifiers, 
said I-frame identifier being associated with said 
stored I-&ame, and each of said P-framc identifiers 
being associated with a corresponding one of said 
stored P-frames; 

storing said I-frame identifier and F-frame identifiers 30 
on said gateway server; 



25 



transmitting said I-frame identifier, said P-frame iden- 
tifiers, said text string, and said descriptive text 
attributes in a control data stream on said broadcast 
communications medium from said gateway server 
to said display device to enable said display device 
to identify said I-frame and said P-frames for de- 
coding and display when they are subsequently 
transmitted m a video data stream on said broad- 
cast communications medium and to enable said 
display device to reconstruct said text block for 
subsequent display thereon; 

transmitting said I-lBrame and said P-frames in said 
video data stream from said video server to said 
display device, each frame being transmitted with 
an associated header containing its corresponding 
identifier to enable its identification thereby; 

decoding, in said display device, each frame from said 
video data stream whose header contains an identi- 
fier matching a corresponding one of the identifiers 
transmitted thereto in said control data stream, said 
decoding being carried out in accordance with said 
MPEG algorithm to reconstruct said background 
and overiay images for display on said display 
device 

reconstructing, in said display device, said text block 
from the text string and descriptive text attributes 
transmitted thereto in said control data stream; and 

displaying the reconstructed background image, 
overlay images, and text block on said display de- 
vice. 

* « * * * 
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